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Controlled  stochastic  dynamical  systems 

1  Singular  Control 

1.1  Problem  formulation 

The  work  described  in  this  section  is  related  to  section  3  of  the  original 
proposal  [21].  Consider  a  weak  solution  to  the  one-dimensional  stochastic 
differential  equation 

n(Xx(s-))ds  +  [  a(Xx(s—))dW(s)  +  A(t)  (1.1) 

Jo 

where  x  is  a  real  number,  \W(t)  :  t  >  0}  is  a  standard  Brownian  mo¬ 
tion  adapted  to  a  right  continuous  filtration  :  t  >  0}  on  a  probability 
space  (fi,  P).  The  functions  [i  and  a  are  differentiable  and  satisfy  the 
assumptions  listed  below.  The  control  process  A(.)  is  {3t  [[-adapted,  right 
continuous  with  left  limits,  and  of  bounded  variation  on  finite  time  intervals. 
Also,  A(0)=0. 


X 


:{t)  =X+  [ 

Jo 


The  main  assumption  here  is  that  the  drift  and  diffusion  coefficients 
satisfy  the  following  two  conditions: 

The  functions  y  and  a  are  continuously  differentiable  on  R,  Av)  <o  _ 
for  all  ?/,  inf  a(y)  >  0  and  x/i(x)  <  0  for  all  x  /  0. 


The  diffusion  coefficient  <r  is  allowed  to  be  unbounded  subjected  to  growth 
conditions 


y(x)  —  x 
a2(x) 


dx 


x  —  y(x) 
a2(x) 


dx  =  oo. 


(1.3) 


In  the  absence  of  the  control  process,  (1.1)  can  be  considered  as  a  random 
perturbation  of  the  stable  dynamical  system  x  =  /i(x)  which  has  a  unique 
global,  asymptotically  stable  equilibrium  point  at  the  origin.  The  cost  func¬ 
tion  C(-)  also  has  its  unique  minimum  at  the  origin  and  it  increases  as  x 
moves  away  from  the  origin. Therefore,  our  study  concerns  the  long  term 
stability  of  a  randomly  perturbed  stable  dynamical  system  with  a  minimal 
control  effort.  Such  situations  are  abundant  in  applications,  such  as  Capac¬ 
ity  Expansion  Theory(see  example  2,  p.393  in  [23],  [19]). 


Let  U  be  the  collection  of  such  processes  (Xx,A)  which  satisfy  (1.1)  to¬ 
gether  with  above  assumptions. We  shall  study  the  ergodic  stochastic  control 
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problem  with  optimal  value  Ao  defined  by 

A0  =  inf  limsup  }-Ex  [  [C(Xx(s))ds  +  d\A\(s)}.  (1.4) 

u  T— >oo  1  Jo 

Notice  that  Ao  is  a  constant  which  is  independent  of  the  initial  value  x. 
In  [25],  we  were  able  to  characterize  an  optimal  control  with  a  Markovian 
state  process  X  that  achieves  the  value  Ao  and  to  relate  it  to  the  value  func¬ 
tions  of  the  family  of  discounted  control  problems  defined  by 

/*oo 

Va(x)  =  inf Ex  /  e~as[C(Xx(s))ds  +  d|A|(s)]  (1.5) 

u  Jo 

as  well  as  to  the  value  functions  of  the  family  of  finite  horizon  control  prob¬ 
lems 


Vq(x,  T)  =  inf  Ex 

lA 


[C'(Xx(a))da  +  d|A|(s)]. 


(1.6) 


The  qualitative  nature  of  the  optimal  policies  depends  on  the  growth  rates 
of  \/j!(x)\  and  \C\x)\.  Therefore,  we  introduce  a  function  H  defined  by 


H{x)  =  n'{x)  +  \C'(x)\  for  all  x  in  R.  (1.7) 

We  were  also  able  to  apply  our  results  of  ergodic  control  problem  to 
derive  a  complete  solution  for  the  following  minimization  problem  with  a 
constraint: 

Let  m  >  0  be  a  constant  and  let  lAm  be  the  collection  of  processes  (Xx,A) 
in  U  which  satisfy  the  constraint 

lim  sup 

T^OO 


E\A\(T) 


<  m. 


(1.8) 


The  constrained  problem  is  the  following: 


1  fT 

Minimize  limsup—  E  /  C(Xx(s))ds  (1.9) 

T^oo  T  ,7q 


over  all  the  admissible  processes  (Xx,A)  in  Un 
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1.2  Main  results  and  their  importance 

In  [25],  we  were  able  to  obtain  the  following  results. 

1.  If  H(x)  <  0  for  all  x,  then  zero  control  policy  is  optimal. 

2.  If  there  exist  constants  e  >  0  and  5  >  0  such  that  H(x)+e/i'  (x)  >  6  >  0 
for  large  |a:|,  then  there  is  a  finite  interval  [a*,  b*]  such  that  the  opti¬ 
mal  state  process  X*  is  a  reflecting  diffusion  on  [a*,  b*].  In  this  case, 
optimal  control  process  A*  is  the  difference  of  local-time  processes  of 
X*  at  the  points  a*  and  b*. 


3.  In  the  both  cases  above,  the  following  Abelian  limits  hold: 


lim  sup  \aVa(x)  —  Ao|  =0 

a_>0 \x\ <K 
for  each  K  >  0. 


and 


lim  sup 

T^°°  \x\<K 


Vo(x,T) 


—  Ar 


=  0 


4.  There  is  a  constant  mo  >  0  such  that  for  all  m  >  mo,  the  constrained 
optimization  problem  described  in  (1.8)  and  (1.9)  has  an  optimal  strat¬ 
egy  of  the  sort  described  in  item  2  above. 

Our  results  in  items  1  and  2  greatly  generalizes  the  works  of  [9]  and  [14]. 
Items  1  and  2  provides  a  near  dichotomy  of  very  different  types  of  optimal 
strategies  on  how  to  control  a  random  perturbation  of  a  stable  dynamical 
system.  Unlike  in  many  other  articles  in  the  literature,  we  make  no  as¬ 
sumptions  on  the  convexity  of  the  running  cost  function.  Our  approach  to 
ergodic  control  problem  is  completely  new  and  we  use  Abelian  limits  and 
this  method  can  be  applied  in  other  situations.  In  an  interesting  article  [1], 
Arisawa  and  Lions  obtained  the  uniform  convergence  of  the  Abelian  limits 
in  item  3  (uniformly  on  the  state  space),  when  the  state  process  is  in  a  com¬ 
pact  space.  Our  results  generalize  their  work  to  processes  on  whole  real  line 
and  our  work  shows  that  uniform  convergence  on  compact  sets  is  the  best 
possible  when  the  state  space  is  non-compact . 

In  a  recent  article,  [2],  Ata  et  al.  considered  a  constrained  minimization 
problem  similar  to  (1.8)  and  (1.9)  when  the  state  space  is  a  fixed  interval 
[0 ,b\.  Their  work  arises  from  an  application  in  Wireless  network  communi¬ 
cation  theory.  Our  work  in  section  6  of  [25]  generalizes  those  results  to  a  non 
compact  state  space.  For  constrained  optimization  problems  with  dynamic 
constraints,  new  methods  were  developed  in  [3]. 
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2  Singular  Control  and  Queues  in  Heavy  Traffic 


The  work  described  in  this  subsection  is  the  result  of  a  joint  project  with 
Arka  P.  Ghosh  in  the  Statistics  Dept.,  Iowa  State  University. 

Optimal  buffer  size  for  queueing  networks  is  a  very  important  design 
consideration  in  engineering  applications  and  there  are  very  few  technical 
results  available  in  the  literature  (see  [2]).  The  following  two  applications  of 
singular  control  theory  address  this  issue  when  the  queueing  networks  are 
in  heavy  traffic. 


2.1  Optimal  Buffer  Length  Problem 

We  consider  a  one  dimensional  singular  control  problem  which  can  be  ob¬ 
tained  as  a  formal  limit  of  a  diffusion  scaled  queueing  network  models.  The 
non-negative  state  process  Xx  which  represents  the  queue  length  process  is 
given  by 

Xx(t)  =  x  -  [  u(s)ds  +  W(t)  +  L(t)-U(t )  (2.1) 

Jo 

where  W{)  is  a  one  dimensional  Brownian  motion  adapted  to  a  right  con¬ 
tinuous  filtration  :  t  >  0}  on  a  probability  space  (fi,#,  P).  The  process 
L(-)  is  the  local-time  process  of  the  state  process  Xx  at  the  origin. 

The  processes  u(-)  and  U (•)  are  considered  control  processes  and  are  adapted 
to  the  filtration  {5t  :  t  >  0}.  The  drift  control  process  u(-)  takes  values  in  an 
a  priori  known  control  set  A  and  this  process  is  analogous  to  the  difference 
between  the  service  rate  and  the  arrival  rate  of  the  queueing  model.lt  is  also 
assumed  that  E  u(s)ds  <  oo  for  each  t  >  0. 

The  other  control  process  U(-)  is  a  non-decreasing  ROLL  process  and  U( 0)  = 
0.  In  the  queueing  model,  controller  is  allowed  to  reject  the  customers  who 
try  to  enter  the  queue  to  keep  the  congestion  cost  low.  In  (2.1),  U(t)  repre¬ 
sents  the  cumulative  number  of  “rejected”  customers  during  [0,  t] .  In  general, 
there  are  no  constraints  on  the  queue  length,  but  the  controller  may  choose 
a  threshold  level  b  >  0  for  the  queue  length  to  control  the  costs. 

We  assume  that  the  cost  functions  h  and  C  are  non-negative,  convex  C 2  func¬ 
tions  and  they  represent  the  congestion  cost  (or  holding  cost)  and  the  control 
cost  as  described  below.  For  a  given  admissible  control  system  (Xx,u,U), 
we  introduce  the  cost  functional  £(T)  by 

C(T)=  fT[h(Xx(S))  +  C(u(s))]ds  +  p-U(T)  (2.2) 

Jo 
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Here,  p  is  a  positive  constant  which  represents  the  penalty  per  rejection.  The 
long  term  average  cost  per  unit  time  is  represented  by  the  cost  functional 

J(Xx,  u,  U)  =  limsup^[C(T)].  (2.3) 

T^oo  1 

The  value  function  of  this  ergodic  problem  is  given  by 

v0=  inf  J(Xx,u,U )  (2.4) 

The  problem  addressed  in  [7]  is  to  find  an  optimal  control  policy  (X*,u*,  U*), 
for  the  control  problem  (2.4)  under  reasonable  assumptions  and  to  charac¬ 
terize  the  value  vq  in  (2.4). 

In  addition  to  the  previous  assumptions,  we  also  assume  that  the  control  set 
A  is  a  subset  of  [0,  oo)  and  it  contains  the  interval  [0,  6q]  where  6 o  satisfies 

c\e  0)  =  i. 

2.2  Main  results  on  optimal  buffer  size  and  their  importance 

1.  We  were  able  to  show  that  there  is  an  optimal  buffer  length  b*  related 
to  the  optimal  policy.  The  controller  should  not  let  the  queue-length  to 
exceed  this  threshold  point  b* .  More  precisely,  we  obtained  a  Markov¬ 
ian  optimal  policy  described  by 

X*(t)  =  x-  [  u*(X*(s))ds  +  W(t)  +  L*(t)  —  U*{t)  (2.5) 

Jo 

Here,  the  optimal  drift  control  u*(X*(t))  is  feed-back  type  and  the 
optimal  rejection  process  U*(t)  is  the  local-time  process  of  X*  at  the 
threshold  b*.  L*  is  the  local-time  process  of  X*  at  the  origin. 

2.  The  value  vq  is  characterized  by  an  equation  which  relates  uo  with  the 
threshold  b*.  The  existence  of  a  unique  C 2  solution  to  the  correspond¬ 
ing  (HJB)  equation  is  also  established  in  this  work. 

3.  An  algorithm  is  developed  to  compute  the  threshold  b* . 

When  the  buffer  size  is  fixed,  and  there  is  no  congestion  cost(i.e.  h(-) 
is  identically  zero),  this  problem  reduces  to  the  work  of  [2].  Also, the 
article  [6] ,  address  a  discrete-time  control  problem  for  a  queueing  sys¬ 
tem  in  equilibrium  and  their  cost  structure  contains  a  control  cost  and 
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a  congestion  cost,  but  there  is  no  rejection  cost.  Our  work  is  mo¬ 
tivated  by  these  two  articles  and  our  cost  structure  incorporates  all 
three  types  of  costs.  The  optimal  policy  derived  here  can  be  used  to 
obtain  asymptotically  optimal  policies  for  large  queueing  networks  in 
heavy  traffic.  For  a  discussion  on  the  use  of  optimal  singular  control 
to  find  asymptotically  optimal  control  policies  for  queueing  networks 
in  heavy  traffic,  we  refer  to  [20]. 

2.3  Dynamic  control  of  a  queueing  network  with  reneging 

This  is  a  project  [8]  which  is  just  completed  and  currently  we  are  in  the 
process  of  preparing  a  first  draft.  This  work  is  motivated  by  our  previous 
work  in  [7]  and  also  by  the  recent  articles  of  [18]  and  [20].  The  model  here 
is  similar  to  that  of  (2.1)  but  in  addition  to  that  impatient  customers  are 
allowed  to  leave  the  queue  at  a  rate  of  7  >  0.  Therefore  with  the  same 
notation  as  in  (2.1),  the  state  process  Xx  satisfies  the  equation 


Xx(t)  =  x  —  f  u(s)ds  —  7  f  Xx(s)ds  +  W(t)  +  L(t)  —  U(t)  (2.6) 
Jo  Jo 

for  all  t  >  0. 

The  control  processes  u  and  U  satisfies  the  same  assumptions  as  in 
(2.1)and  the  drift  control  process  u{-)  takes  values  in  an  a  priori  known  con¬ 
trol  set  A  .  But  in  this  case,  we  address  the  infinite  horizon  discounted  con¬ 
trol  problem.  To  describe  it,  given  an  admissible  control  policy  (Xx,u,U), 
the  discounted  cost  Jp(Xx,u ,  U)  is  given  by 

/•OO 

Jp{Xx,  u,  U)  =  /  e-5s[(P1Xx(s)  +  C(u(s)))ds  +  p-dU(s)}  (2.7) 

Jo 

where  (3  and  6  are  positive  constants.  Instead  of  the  congestion  cost  h(-) 
term  in  (2.2),  here  there  is  a  reneging  cost  due  to  the  departure  of  the 
impatient  customers.  As  in  (2.2),  the  constant  p  represents  the  penalty  rate 
per  rejection.  The  value  function  is  defined  by 


Vp(x)=  inf  Jp(Xx,u,U)  (2.8) 

(■ Xx,u,U ) 


for  each  x  >  0. 

Next  we  introduce  the  constant  po  >  0,  which  is  crucial  in  the  determination 
of  our  optimal  control  policy.  Let 


Po 


Pi 

5  +  7 


(2.9) 
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where  the  constants  /?,  7  and  <5  are  as  above.  Our  main  assumption  here  is 
that  the  control  set  A  is  a  subset  of  [0,  00)  and  it  contains  the  interval  [0,  $o] 
where  9q  is  the  unique  positive  real  number  which  satisfies 


c\e  0)  =  p0. 


(2.10) 


2.4  Main  results  and  their  importance 

1.  When  0  <  p  <  Po,  we  show  that  there  is  a  Markovian  optimal  policy 
and  a  corresponding  threshold  b*  >  0  described  by 

X*x(t)  =  x-  f  u*{X*x{s))ds-  7  f  X*(s)ds  +  W(t)  +  L*(t)-U*(t) 
Jo  Jo 

(2.11) 

Here,  the  optimal  drift  control  u*(X*(t))  is  feed-back  type  and  the 
optimal  rejection  process  U*(t)  is  the  local-time  process  of  X*  at  the 
threshold  b*.  L*  is  the  local-time  process  of  X*  at  the  origin. 
Therefore,  the  threshold  b*  can  be  considered  as  the  optimal  buffer 
length  associated  with  (2.8). 

2.  When  p  >  po,  we  show  that  there  is  an  optimal  Markovian  policy  and 
the  state  process  X*  can  be  described  by 

X*(t)  =  x-  f  u*{X*x(s))ds-  7  f  X*x(s)ds  +  W(t)  +  L*(t)  (2.12) 
Jo  Jo 

Here  L*  is  the  local-time  process  of  X*  at  the  origin  and  there  is  no 
rejection  at  all  and  thus  the  optimal  rejection  process  U*  is  identically 
zero.  Therefore,  the  optimal  state  process  X*  takes  values  in  [0,  00) 
and  thus  there  is  no  optimal  finite  buffer  size. 

3.  The  state  process  of  the  singular  control  problem  is  a  diffusion  limit 
of  a  sequence  of  state  processes  of  queueing  networks  in  heavy  traf¬ 
fic.  Therefore,  as  similar  to  [20],  we  use  our  explicit  optimal  policies 
described  above  to  obtain  asymptotically  optimal  policies  for  the  cor¬ 
responding  queueing  network  control  problems. 
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Discretionary  stopping  and  degenerate  variance 
control 

3.1  Problem  formulation 

The  results  in  this  section  is  joint  work  with  Daniel  Ocone,  Dept,  of  Mathe¬ 
matics,  Rutgers  Univ.,  New  Brunswick.  This  work  is  related  to  the  proposed 
problems  in  section  2  of  the  original  proposal  [21]. 

Consider  a  one  dimensional  stochastic  differential  equation 

X*(t)  =  x+  f  b{X%(s))ds+  [  u(s)dW(s)  (3.1) 

Jo  Jo 

where  x  is  a  real  number,  {W(t)  :  t  >  0}  is  a  standard  one-dinrensional 
Brownian  motion  adapted  to  a  filtration  {$t  ■  t  >  0}  on  a  probability  space 
(f!,S,P).  The  control  process  u(-)  is  progressively  measurable  with  respect 
to  the  above  filtration  and  the  controller  is  allowed  to  choose  it  subject  to 
the  constraint 

0  <  u(t)  <  (To  for  all  t  >  0.  (3-2) 

Here  <7o  is  a  given  positive  constant. 

We  assume  the  following: 

1.  The  drift  coefficient  6(-)  is  a  (^-function,  6(0)  =  0  and  the  origin  is 
the  unique  asymptotically  stable  equilibrium  point  for  the  differential 
equation  =  b(x)  and 

2.  the  running  cost  function  C(-)  is  a  strictly  concave  (72-function  which 
attains  its  unique  maximum  at  the  origin. 

Given  an  adapted  variance  control  process  u  and  a  stopping  time  r  with 
respected  to  the  filtration  {3i  :  t  >  0},we  call  (u,r)  an  admissible  control 
policy  and  let  U  be  the  collection  of  all  admissible  control  policies.  We  define 
the  reward  functional 

J(x,u,t)  =  E  [  e~atC{X^{t))dt  (3.3) 

Jo 

where  a  is  a  positive  constant.  Hence,  the  value  function  is  given  by 

V(x)  =  sup  J(x,  u,  r).  (3-4) 

u 
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3.2  Main  results  and  their  importance 

Control  problems  with  discretionary  stopping  are  addressed  in  the  recent 
literature(  [10] ,  [12] ,  [13]).  In  these  articles,  only  the  drift  coefficient  is  allowed 
to  control  and  the  diffusion  coefficient  is  strictly  non-degenerate.  Importance 
of  our  work  is  due  to  the  following  three  reasons. 

a)  The  diffusion  coefficient  is  controlled  here  while  all  the  previous  articles 
address  only  the  drift  control  problems. 

b)  We  allow  the  controlled  diffusion  coefficient  to  take  the  value  zero. 

c)  We  provide  a  martingale  characterization  (as  similar  to  [12])  for  the 
value  function  for  the  degenerate  variance  control  problem. 

The  state  process  can  evolve  according  to  a  combination  of  a  deterministic 
and  stochastic  behavior.  We  obtain  an  optimal  strategy  when  the  drift  term 
has  the  linear  form  b(x)  =  —6x  where  0  is  a  positive  constant.  Here  are  the 
main  results: 

1.  Let  Q(-)  be  a  non- negative,  bounded  continuous  function  defined  on 
R  and  let  the  initial  point  x  be  fixed. 

(i)  If  Q(X^(tAT))e~a^tAT'>+ CT  e~atC(X^(t))dt  is  a  super-martingale 
for  each  admissible  control  policy  ( u ,  r)  in  U ,  then  Q(x)  >  V(x). 

(ii)  If  Q(-)  satisfies  the  above  condition  (i)  and  if  there  is  a  state 
process  (•)  corresponding  to  an  admissible  control  policy 

(■ u*,t *)  so  that  Q(Z“‘(tAr*))e-Q(iAT‘)  +  f*AT*  e~atC{Zf  (t))dt 
is  a  martingale  and  Q(Z%  (r*))  =  0  on  the  set  [r*  <  oo],  then 
Q(x)  =  V(x)  and  (•)  is  an  optimal  state  process  and  (■ u*,t *) 
is  the  corresponding  optimal  control  policy. 

2.  When  b(x)  =  —9x  where  9  is  a  positive  constant,  the  value  function 
V(-)  in  (3.4)  is  a  C'1-function.  Furthermore,  there  exist  four  points 
c*  <  a*  <  0  <  j3*  <  d*  such  that  the  following  admissible  strategy 
(u*,  t*)  is  optimal. 

(a)  If  x  <  c*  or  x  >  d*  then  choose  r*  =  0  and  stop. 

(b)  If  a*  <  x  <  /3* ,  then  choose  r*  =  oo,  u*(t)  =  0  for  all  t  and  follow 
the  deterministic  motion. 
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(c)  If  (3*  <  x  <  d*  then  choose  u*(t)  =  ao  and  let  f  be  the  first  exit 
time  of  the  process  A“  (•)  from  the  interval  (f3*,d*).  Thereafter, 
follow  as  in  the  steps  a)  or  b)  appropriately. 

In  this  case,  r*  =  r![Xf( t)=d*}  +  00  ‘  I[xf  (f )=/?*]• 

(d)  If  c*  <  x  <  a*  then  choose  u*(t)  =  ao  and  let  f  be  the  first  exit 
time  of  the  process  Xx  (•)  from  the  interval  (c*,  a*).  Thereafter, 
follow  as  in  the  steps  a)  or  b)  appropriately. 

In  this  case,  t*  =  fl[x„*(f)=c»]  +  00  •  )=«*]■ 


4  Controller-and-stopper  game  with 
degenerate  variance  control. 

4.1  Problem  formulation 

The  work  described  in  this  section  is  related  to  the  proposed  problems  in 
section  4  of  the  proposal  [21].  Our  results  are  published  in  [24]. 

We  consider  a  stochastic  differential  game  which  involves  two  players,  the 
controller  and  the  stopper.  The  stopper  selects  the  stopping  rule  which 
halts  the  game,  while  the  controller  chooses  the  diffusion  coefficient  of  the 
state  process.  Our  model  is  described  by  (3.1)  and  the  controlled  diffusion 
coefficient  satisfies  (3.2).  At  the  end  of  the  game,  the  controller  pays  the 
stopper,  the  amount  J^Xx^  e~atC(Xx(t))dt  where  a  is  a  positive  discount 
factor.  We  assume  that  the  drift  coefficient  b  and  the  reward  function  C 
satisfy  the  assumptions  1  and  2  in  the  previous  section.  In  addition,  we 
assume  that  a  —  b\x)  >  0  for  all  x. 

Let  S  be  the  collection  of  all  stopping  rules  and  A(x)  be  the  collection  of 
all  controlled  state  processes  which  satisfy  (3.1).  We  define  the  upper  and 
lower  value  functions  of  this  game  by 

V(x)  =  inf  sup  E  f  e~atC{Xx{t))dt  (4.1) 

T&s  xxeA( x)  Jo 

and 

V_(x)  =  sup  inf  E  I  e~atC(Xx(t))dt  (4.2) 

xxeA(x)  r£S  Jo 

respectively. 

If  V(x)  =  V(x),  then  this  game  has  a  value,  and  in  that  case,  we  denote 
this  common  value  function  by  V(x).  A  pair  (t*,  Z*)  in  S  x  A{x)  is  called 
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a  saddle  point  of  the  game,  if 


[T  rr*(Z*) 

E  /  e~atC(Z*(t))dt  <E  e~atC{Z* (t))dt 

Jo  Jo 

rr*(X) 

<  E 


/  e~atC{X{t))dt 
Jo 


(4.3) 


for  every  r  in  S  and  every  X(-)  in  A(x). 


The  existence  of  a  saddle  point  clearly  implies  that  the  game  has  a  value 
and  in  this  case, 


V{x)  =  V{x)  =  E 


*(z*) 


e~atC(Z*  (t))dt. 


We  intend  to  characterize  a  saddle  point  and  to  derive  the  explicit  form  of 
the  value  function  for  this  game. 


4.2  Main  results  and  their  importance 

Two  player,  zero  sum,  stochastic  differential  games  are  considered  in  [5] 
using  the  viscosity  solutions  framework.  They  show  that  the  existence  of 
the  value  function  when  the  processes  are  non  degenerate  and  the  drift  and 
diffusion  coefficients  are  bounded.  When  the  diffusion  coefficient  is  strictly 
positive  and  only  when  the  drift  is  controlled,  the  controller-and-stopper 
game  is  analyzed  in  [11].  We  refer  to  this  article  for  the  history  of  the 
discrete-time  game  and  references.  In  a  recent  manuscript  [13],  a  martingale 
formulation  is  developed  for  this  game  with  only  drift  control.  In  [24], we 
treat  the  problem  when  the  diffusion  coefficient  is  controlled  and  allowed 
to  degenerate.  Our  work  use  the  methods  we  developed  for  the  degenerate 
control  problems  in  [16]  and  [17].  Here  we  derive  the  value  function  and 
characterize  its  smoothness.  We  describe  the  main  results  below: 

1.  The  stochastic  differential  game  defined  in  (4.1)  and  (4.2)  has  a  value. 
The  corresponding  saddle  point  is  characterized  by  the  existence  of 
two  points  a*  <  0  <  b* .  Let  the  exit  time  of  any  state  process 
Xx  to  be  given  by  the  stopping  time  t*(Xx)  and  the  state  process 
Z*  satisfies  (3.1)  with  the  feed-back  type  diffusion  control  u*(t)  = 
ao I{a*,b*)(Zx(t))-  Then  (t*,u*)  forms  a  saddle  point. 

2.  The  value  function  is  Lipschitz  continuous  but  it  fails  to  be  a  C 1- 
function. 
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